When abatch ofobjects needs tobegrasped with asetof chosen actions, each object is loaded into one of the threads such that the object'scenter is at position [0,0,0.6], The number of PyBullet physics simulation steps required to perform the grasp is only five, andallother computations forthegrasps -e.g. Initially,all touch signals are converted into touch charts using the pre-trainedtouchCNN. In the first part of the deformation process, we pass the image signal through a VGG-likeCNN with networkarchitecture described inTable6toextract image features. In the touch only setting, a collection of76 vision charts (here referred to as touch charts) is formed in the shape of a sphere and the vertices on the borders of these charts share edges with each other to allowcommunicationbetweenthem.

Our framework enables the development of the first fully data-driven solutions to active touch on top of learned models for object understanding.

To do so, we introduce a dataset of simulated touch and vision signals from the interaction between a robotic hand and a large array of 3D objects.